I think the idea you're getting at here is that you should be able to point your copy engine at any point in the file system, have it do the "right" thing based only on the information it's provided by copyfile, not external knowledge (like the full source path). [...] That's a lovely idea, but for better or worse, macOS just does not work that way. [...] but if you want to work with the "full" system... Well, I don't think there's any way to do without grappling with a whole bunch of edge cases. Well I did not directly point copyfile at .DS_Store and tell it to do the thing. I started a recursive copy on a folder, which happens to have a .DS_Store file in it. And on macOS just about every folder the user views in Finder will have one of those hidden in there, unfortunately. The folder I was copying was on a NAS, has like 24 videos in it. I'm doing this intentional slow copy to test everything out. So the copy gets like 98% done and then it needs a diaper change on the .DS_Store file! I'd ignore all .DS_Store errors and, potentially, just skip copying them all together. I have no intention of including .DS_Store in the copy. I am ignoring it now. I'm not really sure if they should even exist and probably shouldn't passively travel in a copy (maybe with the one exception when an advanced user is viewing hidden files and actively puts it in the pasteboard). It's okay with me if I can't do the copy on DS_Store, I guess. I generally want my app to do what the user is trying to do as long as it can't hurt them. My main issue is it doesn't report the reason why I can't do the copy. In fact sometimes I can do the copy but similar to the issue in the other thread it seems that when the .DS_Store is on the NAS, copyfile fails and that may have something to do with it. But I can copy .DS_Store that are on the local drive (perhaps because of the clone option I dk). And yes Finder can copy it, even from the NAS. So I guess if I really wanted to copy it I might be able to dust off that old Carbon File Manager (but I actually don't). Mainly my concern is cryptic failures even though I am primarily dealing with user facing files in reasonable user-facing locations. I'm not really trying to travel to the underworld. In any case if my app doesn't have the right privileges to copy .DS_Store or whatever (which are littered all over the file system) a normal error, even one with no recovery options seems appropriate. I really want to know if it failed because of something else and not permissions (perhaps because the way the data is stored). If there is another file I come across that stores data in a similar way...and the user would reasonably WANT to copy that file, that's actually what I'm more concerned about than .DS_Store.

Just read your reply again more carefully as it is not the end of the day and I'm not tired :) . As the thread up above shows, the primary way to bypass the issue "copyfile" is having is to use the "..namedfork/rsrc" convention, Interesting. I was using xattr API. So just skipping the resource fork during copyfile, then running copyfile again over the src and destination (with .namedfork/rsrc convention) seems to work on my dummy test file. Is that expected to be a reliable workaround? I guess if this actually works reliably I wouldn't have to restart copy from scratch or suppress deprecated API warnings. Or maybe there good reason to start the copy from scratch with the Carbon file manager?

Have you filed a bug on this and, if so, what is the bug number? Not yet, but I do have another thread where I describe the same problem in regards to .DS_Store: https://developer.apple.com/forums/thread/819041 I will post the bug number (most likely there) when I carve out a little time for it. Currently if copyfile fails and errno is not set my code falls back to a kind of 'Unknown Error' which I hate but if the user is counting on my app to perform the copy it's kind of a gamble if the operation doesn't succeed to just hope not critical data was lost. A big part of this is that xattrs aren't actually "safe“— the system doesn't have a documented system for defining how they should be handled, and app/copy engine/file format support for them is inconsistent, so storing critical data in them can end up being a good way to lose critical data. Importantly, that kind of data loss isn't really a "bug" in the same way other kinds of data loss would be. The general "point" of xattrs is to allow the attachment of arbitrary metadata, but part of that contract is that it's not really part of the "core" file data. This was always my understanding of xattrs. The resource fork WAS originally intended as a larger-scale data storage location, which meant repurposing it was substantially less disruptive (and required a lot less work) than creating a new "bulk" xattr would have been, particularly in the context of HFS+ (where they were originally architected and introduced). Yea my concern is if these files are really old the user may be keeping them around for a long time because they consider them to be important. So I wouldn't want to be blamed if they copied the file in my app, discard the original, and the copy is corrupted. Like the Carbon framework, resource forks were before my time though. The answer depends on what you're trying to do. Basically what I'm trying to do is avoid failure. If I can reasonably and gracefully handle the issue and not present an error, that is best. At the same time, I feel a responsibility to the user. If I don't show an error, the copy should have worked and I don't want to silently hide true errors. Another less important need (but still kind of important), is to keep the implementation as simple as possible. I don't want to rewrite copyfile as I already got like three billion lines of code in my app and I have tons of other stuff to tackle. I don't see much return on investment doing that. Basically my workaround currently is if I run into a resource fork in the copyfile callback, I set a flag and skip it. Then when copyfile finishes (if successfully) I just copy the resource fork over from source to destination. If that sounds like a bad idea, let me know and maybe I'll just fallback to the Carbon file manager instead.

Sorry for the double post. But just a quick additional question. Am I better off working around the issue by retrying with the Carbon file manager or should I just skip the resource fork in the copyfile callback, then copy it over myself at the end?

I'm experiencing this issue. Copy file works locally for these files but fails when coming from a NAS. Works from Finder in both cases, as previously described above. copyfile does not seem to provide a usable error. The callback is invoked twice and I get : what == COPYFILE_COPY_XATTR stage == COPYFILE_START Then the callback is invoked a second time with: what == COPYFILE_COPY_XATTR stage == COPYFILE_FINISH In both cases my callback function returns COPYFILE_CONTINUE. Then the copyfile function returns -1, errno is not set. Not getting a usable error unfortunately forces kind of jackhammer workarounds. Have you noticed this issue for files without resource forks (but use other xattrs) or is the resource fork the main cause?

That does the thing! Thanks! And, yes, we probably should have documented all of this, so PLEASE file about this and post the bug number back here. I filed FB22260532

I tried setting the creation date to: 1946-02-14 08:34:56 +0000 right after creating the folder. but doesn't have the desired effect. Maybe I didn't convert the date correctly?

Thanks for the info! I've been split trying to finish up a different piece of my app related to this. I briefly tried to get this to work today using the old Carbon API which I must confess I'm not super familiar with (before my time) but didn't have any success. Before I posted this thread I tried what looked like the most straightforward way to accomplish this: [fileManager createDirectoryAtURL:someURL withIntermediateDirectories:NO attributes:@{NSFileBusy:@(YES)} error:&error]; But no luck.

Thanks a lot for explaining that. It's good to know that I can do this but I'm still hesitant to use it given the 'strongly discouraged' language in the documentation. Does the 'strongly discouraged' stuff only apply to clonefile function or should I avoid 'cloning' directories in general with the copyfile function?

I just found that Quinn wrote about this a bit here: https://developer.apple.com/forums/thread/784446 I'm still unsure why the man page for one function directs you to use another function - but that man page says cloning directories is unsupported so I'm still a bit confused about the recommendation. Is cloning directories unsupported or strongly discouraged? Basically I have to move off NSFileManager because it doesn't provide progress and it doesn't support cancelling. But I'd like to clone when possible for speed and to save the user disk space.

Thanks for all the information! The other problem here is that you're assuming the APIs involved here operate on paths... when they don't. Perhaps I didn't explain myself well. I wasn't making assumptions about how the the APIs operate - the only thing I was claiming (well trying to) was that paths provide the concept of file location and in certain contexts that concept matters to GUI apps (with the NSBrowser example being an obvious one). The current API for file reference urls that I'm aware of don't try to solve that problem in any way. Note that this INCLUDES FSEvents. [...] If the directory you're monitoring moves... then FSEventStreamCreate will happily continue sending you events about that directory, NOT the original path location. If you want to know when the directory you're monitoring moves, then that's what "kFSEventStreamCreateFlagWatchRoot" Very interesting. Yeah I use that (watch root) With the watch root flag set it does seem to just track whatever is at the path. To test this I just put 'TestFolder' on my Desktop and started a FSEventStream on it. Then I rename 'TestFolder' to TestFolder2 I pick up the root change. If I start making changes in TestFolder2 I'm not picking up any changes. With the stream still open, I create a new folder 'TestFolder' - and the opened stream starts picks up changes on the new TestFolder (not TestFolder2) along the original path - so it does 'stick' to the path and doesn't follow the reference (I've never tried not using kFSEventStreamCreateFlagWatchRoot since I need it). then it might be worth starting a new thread that's focused on those issues. I've not had problems with FSEvents I just brought it up because if there was some pathless way to achieve what an FSEventStream provides I'd be open to exploring it. I'll be sure to open a new thread if I run into anything.

Revisiting this six years later - moving to copyfile instead of using NSFileManager. Nothing seems to have changed unfortunately. FB7656661 is still open.

Actually if I create the alias in my app using -bookmarkDataWithOptions:includingResourceValuesForKeys:relativeToURL:error: it resolves proper. so it appears to be an issue with the way Finder makes aliases. Maybe they are using relativeToURL: parameter? Weird.

It means apps don't need to actively track the current location of their files, but can instead determine its current location whenever the user actually asks for it. What should happen if you expand a directory with four subfolders in a NSBrowser (so four columns). And the third folder moves? If an app just follows the file reference URL the UI is broken. I mean if there are/were APIs that handled this in file reference world I'm with you! The concept of file location matters to users and to a lot of apps. I hope Apple doesn't want to put apps in single file jail. Here you get that one txt document and if it moves don't worry we got you. There are other types of apps that do other things. Please don't turn macOS into iOS :)

I'm not sure what you mean here. If you want to know the current path location, then you ask the kernel to convert the reference to a path. [..] I'm not quite sure what you're thinking of here. If you're trying to display the contents of a directory hierarchy, then the process is exactly the same for paths and references. You start with the container and recursively iterate its contents, displaying whatever you find. What about changes? FSEventStreamRef reports changes on a directory. Does it work on file reference urls? I never even thought of trying. I can pick up changes on a subfolder 4 folders away from the root folder I'm watching, all using a single FSEventStreamRef. I'm not saying it isn't possible I just don't know? how can I parse a single change using current APIs in a deep tree with opaque file reference urls that don't have a concept of location without first converting to path to see what level I'm at? Sure. Why not? That's what the Carbon File Manager did. Well I'm not totally against the idea nor was I arguing that it was impossible. I'm just saying it isn't in the API today, so I need a path to create a file. As you already mentioned previously macOS is very path based. In my original sample the code does not indicate in any obvious way that I wanted to use a file reference URL but merely asking NSFileManager the 'relationship' caused a really undesirable side effect. but I suspect this is actually a new bug, possibly introduced in Swift Foundation[1] [1] Much of the Objective C "version" of Foundation is actually written in Swift now. Lord have mercy! I have another issue that may or may not be related to this in some way but I think I'm going to have to spin up a separate thread.